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Conformance Resources and Information 



Conformance is usually defined as a way to determine if an implementation 
faithfully meets the requirements of a standard or specification. Determining 
whether a product faithfully implements a standard is essential to creating robust, 
interoperable solutions. The NIST/ITL Conformance Pages provide an online 
reference to documents, presentations, test suites related to conformance, 
conformance issues, and conformance-related topics. 



General Conformance 

• W3C Specification Guidelines 

Draft guide on writing better specifications. (This link takes you of the NIST 
site) [Nov 2004]. 

• Principles for writing conformance requirements [pdf] 

List of principles to help craft conformance requirements (aka test assertions) 
[Rosenthal, Lyon; Feb 2005]. 

• Writing Quality Specifications 

Presentation on how to write a better specification. [Skall, July 2004]. 

• What is this thing called conformance? 

An overview of conformance including a discussion of terminology, 
methodology, and benefits. Additionally, various test suites for XML 
technologies are presented. [Rosenthal, Brady; NIST/ITL Bulletin, January 
2001]. 

• Presentation: What is this thing called conformance? [ppt] 
Presentation on conformance including what it is, how it works, and what the 
benefits are. Additionally specific examples from available conformance test 
suites (XML, DOM, XSLT) are presented. [Rosenthal, Brady, XML 2000 
Conference, December 2000]. 

• Conformance [ppt] 

Presentation given at the ebXML Tokyo meeting, November 2000. An 
overview of conformance is given, including the topics and issues that need to 
be addressed when including a conformance clause in a specification. [Skall, 
Rosenthal, November 2000]. 

• Conformance Testing 

With any standard, eventually the discussion turns to "how will we know if an 
implementation conforms to the standard?" This paper defines conformance 
and conformance testing as well as describes the components of a 
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conformance testing program. [Gray, Goldfine, Rosenthal, Carnahan]. 

• Overview of Conformance Testing 

Discussion of conformance terms, assessment program, types of testing, and 
factors for success. [Gray]. 

Validation and Certification 

Many of the documents and presentations in the General Conformance section 
include a discussion of validation and certification programs 

• Conformance Testing and Certification Framework [pdf] 

This paper presents general concepts, components, and issues related to 
establishing and administering a conformance testing program. Although it 
addresses testing for ebXML specifications, it can be applied to testing any 
specifications. [Rosenthal, Skall, Carnahan, White Paper for ebXML, April 
2001], 

• Conformance Testing and Certification Model for Software 
Specifications 

This paper identifies the key roles, activities, and products involved in 
creating conformance testing and certification programs. [Carnahan, 
Rosenthal, Skall, ISACC '98 Conference, March 1998]. 

Test Methodologies 

• DOM Test Suite Methodology [pdf] 

This paper provides a description of the Document Object Model 
(DOM) test suite methodology for how conformance tests to the W3C 
DOM specification was developed using automated test generation 
techniques. [NIST GCR04-860, 2004]. 

• Interactive Conformance Testing for VRML 

Paper describing the test development strategy and design issues in 
developing the NIST Virtual Reality Modeling Language (VRML) 
conformance test suite. [Brady, Rosenthal]. 

• Conformance Testing of Object-Oriented Components specified by 
State/Transition Classes [pdf] 

This paper provides a conformance testing technique for new 
components to be integrated into an existing software system. The 
approach relies on finate state machines, database modeling and 
processing techniques, and algorithms for analysis and traversal of 
directed graphs. [Gallagher, 2000]. 

• Formal Specification Languages in Conformance Testing 
Report assessing the potential the advantages of applying formal 
specification languages rather than natural language, to the problems 
of conformance testing.[Liburdy, Gray, Rosenthal, Software Quality 
Week, 1998]. 
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Test Suites 

• XML - A Conformance Test Suite, jointly developed by NIST and W3C 
for the W3C XML and Namespace Recommendations. 

• DOM Conformance Test Suites, developed by NIST and W3C, for 
DOM Level 1 and Level 2, CORE and HTML modules. 

• A. XSL - Conformance Test Suites, developed by NIST, for both 
Formatting Objects (XSL-FO) and XSL Transformations (XSLT/XPath). 

• XML Schema - Conformance Test Suites, developed by NIST, for XML 
Schema Part 2: Datatypes. 

• XML Query - Conformance Tests, developed by NIST, for Functions 
and Operators, Working Draft. 

• ebXML - Test Framework and Conformance Tests for ebXML 
Messaging. 

• Smart Card - BSI and Card Edge Conformance Tests for C and Java 
Bindings. 
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